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Abstract 

This report describes a classroom artifact measure and presents early findings from an 
efficacy study of WestEd’s Reading Apprenticeship (RA) professional development 
program. The professional development is designed to teach high school teachers how to 
integrate subject-specific literacy instruction into their regular curricula. The current RA 
study is notable in that it is the first to include random assignment in its design. The 
National Center for Research on Evaluation, Standards, and Student Testing (CRESST) 
designed a teacher assignment instrument to address the question of whether treatment 
teachers demonstrate greater integration of literacy into their instructional practice than 
control teachers. Early findings based on preliminary data from participating history 
teachers indicate that treatment teachers outperformed control teachers on 6 out of 11 
rubric dimensions. These dimensions address opportunities for reading in the assignment, 
the strategies in place to support successful reading, teacher support for reading 
engagement, and student feedback. Data collection will conclude at end of the 2008- 
2009 school year, followed by a final report. 

Introduction 

This report describes the development and use of a new, artifact-based measure of 
teachers 4 instructional practice. Developed as part of a randomized field trial of the Reading 
Apprenticeship (RA) professional development program, the assignment-based measure 
represents a generalizable strategy for using teacher assignments as a classroom 
implementation measure. The assignments, with associated explanatory materials and student 
work, present a window into teachers’ instructional practice and their analysis can be 
customized to reflect an intervention’s particular emphases. In the sections that follow, we 
start with the context of the RA intervention and background on the assignment instrument 
and then describe the field test methodology, and preliminary results on instrument reliability 
and sensitivity to program effects. 

The RA professional development is designed to teach high school history and biology 
teachers how to incorporate content-specific literacy instruction into their practice, with the 
ultimate goal of improving student literacy and content understanding. RA has been 
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evaluated in the past (Greenleaf, Schoenbach, Cziko, & Mueller, 2001; Greenleaf, Brown, & 
Litman, 2004; Schoenbach, Greenleaf, Cziko, & Hurtwitz, 1999), but this is the first 
randomized study, and few literacy studies that specifically focus on high school content area 
classrooms (Calfee & Miller, 2004; Moje et ah, 2004), as we do here. 

While the study utilizes a range of both teacher and student data, this report describes 
the CRESST-designed teacher assignment instrument. Additionally, we present early 
reliability and sensitivity analyses. Data collection is still in progress and will conclude at the 
end of 2009, at which point our final report will be produced. 

Background 

The RA intervention was developed to improve adolescent literacy levels by increasing 
content area teachers’ ability to integrate literacy instruction into their regular curriculum. 
Historically more policy and instructional attention has been paid to reading in elementary 
grades rather than supporting continued adolescent literacy gains (Biancarosa & Snow, 2004; 
Durkin, 1978). A RAND Reading Study Group report (Snow, 2002) also indicated that 
reading growth for middle school and high school students slows, sometimes significantly. 
This inattention to continued literacy instruction has had detrimental consequences for 
American adolescents. For example, recent National Assessment of Educational Progress 
(NAEP) testing demonstrated that 25% of students tested did not have basic literacy skills, 
while only a small minority had achieved the levels of advanced literacy necessary for 
success in the information-focused economy of the 21st century. (Donahue, Voelkl, 
Campbell, & Mazzeo, 1999; Mullis et al., 1994; NAEP, 2002, 2003). Without adequate and 
advanced literacy levels, all students will struggle to attain content-area knowledge and 
comprehension. 

Unfortunately, non-language arts high school teachers often lack the needed 
instructional strategies, resources, and knowledge about reading comprehension to 
effectively support adolescent literacy development (Alvermann & Moore, 1991; Duffy et 
al., 1987; Duke, 2000; Fielding & Pearson, 1994; Richardson, 1994; Snow, 2002). WestEd’s 
RA instructional framework targets this need (Greenleaf & Katz, 2004; Greenleaf & 
Schoenbach, 2001, 2004). Research has demonstrated that even when teachers are trained in 
using reading comprehension strategies, they still struggle to fully integrate them into their 
content-area instruction. (Alvermann & Moore, 1991; Duffy et al., 1987; Duke, 2000; 
Fielding & Pearson, 1994; Richardson, 1994; Snow, 2002). 

The RA intervention thus includes a key professional development component to help 
teachers bridge the gap between training and classroom implementation. Previous research 
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has demonstrated that students in RA classrooms make positive gains in both reading 
achievement and academic engagement (Greenleaf et ah, 2001; Greenleaf, Brown, & Litman, 
2004; Schoenbach et ah, 1999). 

For this study, the intervention began with a 5 -day training session on the RA 
instructional framework led by a trained consultant during the summer before the first 
implementation year. At the start of the new school year in the fall, teachers began to 
implement the RA instructional framework in their history classrooms. After the first year of 
implementation before the start of school, teachers also met for a 2-day regional video 
conference in which they discussed lesson planning for how to best integrate RA strategies 
and resources into their curriculum. Teacher assignment collection began 1 year after the first 
implementation year. 

Description of Assignment Instrument and Scoring Rubric 

Instrument 

Although teacher surveys and/or classroom observations are most often utilized to 
evaluate instruction, both of these measures have limitations. Teacher surveys are efficient 
and cost-effective and are therefore often used for large-scale studies. However, they rely on 
self-reported data, which can be inaccurate (Mayer, 1999; Spilanne & Zeuli, 1999). On the 
other hand, classroom observations can provide a detailed picture of what is happening in the 
classroom, but due to their high cost, they are typically limited to small-scale studies 
(Matsumura, Gamier, Pascal, & Valdes, 2004). Using teacher assignment ratings to assess 
instructional practice provides an efficient, economically viable, and reliable alternative 
(Aschebacher, 1999; Clare, 2000; Matsumura, Gamier et al., 2002). Previous CRESST 
research in conjunction with the Los Angeles Unified School District (LAUSD) 
demonstrated the effectiveness of using teacher assignments as a method to measure teacher 
practice. (Aschbacher, 1999; Clare, 2000). Further CRESST research demonstrated the 
reliability and validity of the teacher assignment analysis method. (Clare & Aschbacher, 
2001; Matsumura, 2003). 

The teacher assignment instrument for this study includes a cover sheet (see Appendix 
A) with prompts highlighting various aspects of the lesson design (e.g., standards addressed, 
texts included, opportunities for pair and group work, etc.). To supplement the completed 
coversheet, teachers submit samples of high, medium, and low student work in addition to 
handouts and/or texts they made available to students. Both control and treatment teachers 
are required to submit two assignments, one from the beginning of the school year on the 
topic of Industrialization, Immigration, and Urbanization; and one from the end of the school 
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year on the topic of World War II. To identify teacher assignment topics, CRESST and 
WestEd surveyed participating teachers to determine which topics were most often taught at 
the start and end of the school year. 

CRESST teacher assignment rubrics were originally designed to measure the quality of 
teacher instruction. We modified these rubrics to measure RA implementation with a focus 
on three constructs: (a) literacy instruction, (b) content instruction, and (c) monitoring 
student learning (see Figure 1). Each construct reflects various aspects of the professional 
development program. 

The RA Framework directly informed the literacy instruction construct design 
(Greenleaf & Schoenbach, 2004; Greenleaf, Brown, & Litman, 2004). This construct focuses 
on opportunities to read during the lesson, specific strategies utilized to make reading 
successful, and teacher support. Given the professional development’s emphasis on literacy 
improvement, this construct is the most significant in the rubric. 



Constructs 


Literacy Instruction 


Content Instruction 


Monitoring Student Learning 




Measures 





Reading Opportunities 

Reading Comprehension 
Strategies 

Metacognitive Processes 

Disciplinary Reading 

Collaborative Meaning Making 

Support for Reading 
Engagement 

Accommodations for Reading 



Cognitive Challenge 

Support for Cognitive 
Challenge 



Adjusting Instruction 
Student Feedback 



Figure 1. Teacher Assignment constructs and measures. 



The content instruction construct measures the challenge level students were required 
to meet when engaging with history content. Additionally, this construct includes teacher 
support for history content learning. Instruction that provides high academic challenge 
(Marazano, Pickering, & Pollock, 2001; Matsumura, Patthey-Chavez, Valdes, & Gamier, 
2002; Matsumura, Gamier, et al., 2002; Applebee, Langer, Lystrand, & Gamoran, 2003; 
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Stazesky, 2004; Mant, Wilson, & Coates, 2007), incorporates a practice of analytical 
questioning to elicit higher-order thinking (Hattie, 1992; Applebee et al., 2003; Stazesky, 
2004; Mant et al., 2007), and focuses on metacognitive skill building (Haller, Child, & 
Walberg, 1988) and supports student achievement. 

Finally, the monitoring student learning construct measures general teacher practice 
that impacts learning. First, it depicts how teachers use observations about student learning to 
make instructional adjustments (Bulterman-Bos, Verloop, Terwel, & Wardekker, 2003; Pijl, 
1992; Sanders, 1989). Second, this construct addresses how teachers use formative and 
summative assessments as opportunities to provide students with feedback (Hattie, 1992; 
Marazano et al., 2001; Matsumura, Patthey, et al., 2002). 

History Teacher Assignment Rubric 

The following section depicts the qualities of the History Assignment rubric’s 11 
dimensions (see Appendix B for complete rubric). All rubric dimensions were scored on 
4-point scales; 1 was the lowest score point and 4 was the highest score point. 

Reading Opportunities. The purpose of this dimension is to evaluate the degree to 
which the teacher used this assignment as a vehicle to provide students with the opportunity 
to read history texts. Qualities of reading opportunities include: the role of reading, duration 
of reading, and text variety (e.g., essays, arguments, and primary source documents such as 
letters, newspapers, and comic strips). 

This dimension was considered evidence in three domains: Centrality, Time-on-Task, 
and Text Variety. Centrality, the most heavily weighted criterion, considers how central 
reading is to the overall assignment. This is determined by evaluating how the reading task 
related to other aspects of the assignment, whether or not reading was necessary to complete 
the assignment, and how well the text(s) related to the standards, knowledge, and/or skills 
targeted by the assignment. Time-on-Task considers whether an appropriate amount of time 
was set aside for the reading task (e.g., with opportunities for recursive readings of shorter, 
easier texts and/or in-depth reading of longer, more difficult texts). Text variety considers the 
types of texts the teacher provided for the students. Assignments could still receive a score 
point of 4 if only one history text was used, but the other criteria were met. 

Reading Comprehension Strategies. The purpose of this dimension is to describe the 
degree to which the teacher provided students with the opportunity to utilize various 
strategies to assist in their comprehension of history reading. These reading comprehension 
strategies include: generating questions, previewing text organization, and using reading 
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logs. Raters paid particular attention to evidence of teachers utilizing RA-specific reading 
comprehension strategies. 

This dimension was scored using four criteria: The quality and extent of Description, 
Time-on-Task, Purposefulness, and Accountability. Description considers whether or not the 
teacher explicitly calls attention to strategies used during the assignment in addition to how 
clearly the teacher describes the strategies. If the teacher made no mention of them in the 
coversheet but there was evidence of their use in the samples of student work, assignments 
still received some credit for reading comprehension strategies. Time-on-Task considers the 
degree to which sufficient time was set aside for students to use these strategies. 
Purposefulness considers to what extent specific strategies were tailored to specific texts 
and/or reading tasks. Finally, Accountability considers whether or not students were held 
accountable for utilizing strategies (e.g., in one assignment students had to submit their text 
annotations and the teacher provided them with feedback). 

Metacognitive Processes. The purpose of this dimension is to evaluate the degree to 
which the teacher used this assignment as a vehicle to provide students with the opportunity 
to utilize various metacognitive thinking skills, made evident through student annotations of 
text, metacognitive reading logs, and teacher instructions to students to think aloud or to 
discuss thinking and problem solving processes. 

This dimension was scored using three criteria related to executive control: Identifying 
Confusions, Self-Evaluation, and Making Adjustments . Identifying Confusions considers 
whether students had the opportunity to identify challenging material while engaging with 
text (e.g., identifying unknown vocabulary, confusing history concepts, etc.). Self-Evaluation 
considers whether students were required to assess their understanding of history text as a 
formal part of the assignment. Making Adjustments considers the degree to which students 
made specific changes to their approach to a history reading task. 

Disciplinary Reading. This dimension considers the degree to which the teacher used 
this assignment as a vehicle to provide students with the opportunity to utilize Disciplinary 
Reading processes such as comparing and contrasting texts — including maps, graphs, history 
symbols, as well as written text, evaluating the sources of a documents, identifying the 
perspectives or points of view taken, and placing primary source documents into historical 
context. 

This dimension was scored using two criteria: Frequency and Depth. Frequency 
considers how many types of Disciplinary Reading opportunities the assignment provided, 
while Depth considers whether these opportunities were substantial or cursory. 



6 




Collaborative Meaning Making. The purpose of this dimension is to describe the 
degree to which the teacher used the assignment as a vehicle to provide students with the 
opportunity to participate in discussions about history texts. This dimension also considers 
the opportunity teachers provided students to read in small or paired grouping configurations. 

It was scored using four criteria: Purpose, Routine, Accountability, and Connection. 
Purpose considers whether there were clear objectives for the collaboration and if these were 
communicated to the students. Routine considers how formal the collaboration was (i.e., 
collaboration was required of all students, teacher communicates explicit directions about 
how to collaborate, students provided with graphic organizers to guide work, etc.). 
Accountability considers if students were held accountable for work completed during the 
collaboration collectively and/or individually. Connection considers the degree to which the 
collaborative task supports the next step(s) in the overall assignment. 

Teacher Instruction: Support for Reading Engagement. The purpose of this 
dimension is to evaluate the degree to which a teacher supports students in their successful 
completion of the reading task. Specifically, this dimension considers literacy support 
activities such as whether the teacher models (i.e., demonstrates an aspect of the reading 
process), provides explicit instruction (i.e., articulates the various steps and/or processes 
students required of the reading task), provides resources (e.g., consumable texts, graphic 
organizers), and/or establishes literacy routines (i.e., puts in place ongoing and specific 
reading process practices). Teacher support for reading engagement may focus on any aspect 
of the reading process (e.g., reading logs, focused conversations, reading comprehension 
strategies, metacognitive activities, etc.). 

This dimension was scored using two criteria: Frequency and Depth. Frequency 
considers how many types of support the teacher provided, while Depth considers whether 
these supports were substantial or cursory. 

Teacher Instruction: Accommodations for Reading. The purpose of this dimension 
is to describe the degree to which a teacher tailored the assignment to meet the various 
reading needs of his/her students. Specifically, this dimension considers whether the teacher 
differentiated instruction though accommodations such as: providing various texts for 
students to read at different reading levels; providing extra support for struggling readers and 
English Learners (e.g., by modifying instruction, giving help outside of class, and adapting 
the assignment content); allowing students to work at their own pace; and paring struggling 
reading with stronger readers. 
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Rubric. This dimension was scored using two criteria: Frequency and Clarity. 
Frequency considers how many types of accommodations the teacher provided and Clarity 
considers the degree of specificity that teachers described accommodations. 

Cognitive Challenge. The purpose of this dimension is to describe the degree to which 
teachers required students to apply complex cognitive skills when engaging with history 
content and concepts in this assignment. The dimension also considers the level of critical 
thinking teachers required of the students in order to complete the assignment (e.g., problem 
solving, analyzing, and synthesizing information). Specifically, this dimension considers the 
opportunity teachers provided students to construct or transform knowledge as opposed to 
simply recalling, describing, or identifying basic information. 

This dimension was scored using Bloom’s Taxonomy as a guide (Bloom, 1956). ffigh- 
scoring assignments significantly utilized higher-order thinking skills by engaging students in 
reasoning processes such as analysis, synthesis, and/or evaluation of historical concepts in 
order to complete the assignment. These higher-order thinking processes are the means by 
which deep content understanding is acquired. At the other end of the spectrum, low-scoring 
assignments engaged students in basic comprehension skills such knowledge recall, 
definition, and/or labeling of historical concepts in order to complete the assignment. These 
lower-level thinking processes are the means by which surface content understanding is 
acquired. 

Teacher Instruction: Support for Cognitive Challenge. The purpose of this 
dimension is to describe the degree and quality of support a teacher provides for the 
assignment’s cognitive challenge. Specifically, this dimension considers the degree of 
support for the thinking skills (e.g., knowledge, comprehension, application, analysis, 
synthesis, and/or evaluation) and processes that are provided by the teacher for successful 
completion of the assignment. An assignment given a high score on this dimension had to 
demonstrate that the teacher provided support that was focused on the cognitive task students 
were to carry out. Additionally, the assignment most likely had a high percentage of students 
performing at an adequate level or above. 

This dimension was scored using four criteria: Thinking Processes, Structure, 
Peer/Expert Knowledge, and Resources. Thinking Processes considers whether the teacher 
explicitly taught the thinking processes necessary to meet the cognitive challenge of the 
assignment (e.g., through modeling, class discussion, etc.). Structure considers the degree to 
which the cognitive task was broken down into clear and explicit steps. Peer /Expert 
Knowledge considers whether the teacher enabled students to draw on peer or expert 




knowledge during the assignment. Finally, Resources considers whether the teacher made 
materials available to students that would aid in meeting the assignment’s cognitive 
challenge (i.e., samples of student work with critical thinking processes made explicit). 

Monitoring: Adjusting Instruction. The purpose of this dimension is to capture the 
degree to which the teacher adjusts instruction based on monitoring student progress. 
Specifically, this dimension considers whether the teacher made curricular, instructional, or 
lesson adjustments for the immediate benefit of the current students. 

This dimension was scored using two criteria: Specificity and Student Benefit. 
Specificity considers how clearly the teacher articulated the adjustment made. Student Benefit 
considers which students will benefit from the adjustment (i.e., the current class or future 
classes) and how quickly the adjustment is implemented (i.e., during the execution of the 
assignment, some unknown time in the future, etc.). Teachers who wrote about monitoring 
students but made no adjustments received low scores for this dimension. 

Assessment: Student Feedback. The purpose of this dimension is to capture the 
degree to which the teacher provides students with feedback. Specifically, this dimension 
considers whether teachers provided feedback to positively impact student performance. 

This dimension was scored based on the specificity of the feedback. Teachers who gave 
students specific feedback that had the potential to improve student performance during the 
current assignment or in the future received higher scores. Teachers who gave unclear 
feedback or only assigned student work a number grade received lower scores. All feedback 
was considered whether verbal or written, during the assignment or after the assignment. 

Field Test Methodology 

Sample 

The study targeted high schools in 55 California school districts historically serving 
low-performing student populations: There are currently 47 districts actively participating in 
the study. By focusing on low-performing students, this study will ultimately determine 
whether RA can address the pervasive achievement gap between mainstream and minority 
populations (Gee, 1999; Jencks & Phillips, 1998; Snow, Bums, & Griffin, 1998). Schools 
served as the unit of randomization and were matched prior to randomization using five 
factors: 

(1) African-American enrollment, 

(2) English Learner enrollment, 

(3) Latino enrollment, 
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(4) socio-demographic profile, and 

(5) academic performance (independent of socio-demographic characteristics). 

WestEd identified two history and two biology teachers from each participating school and 
then randomly assigned the schools to treatment and control conditions. In order to control 
for teacher experience, all participating teachers are credentialed and have taught for at least 
3 years. 

Data collection began during the 2007-2008 school year with a cohort of history 
teachers (see Table 1). A cohort of biology teachers and a second cohort of history teachers 
are currently in their data collection year. This report reflects preliminary results from 
History Cohort 1 only. History Cohort 2 teacher assignment data will be added to the current 
data set to allow for further analysis. 



Table 1 

Data collection schedule 



Subject 


Initial training workshop 


Practice and follow-up 
training year 


Data collection year 


History Cohort 1 


Summer 2006 


2006-2007 


2007-2008 


History Cohort 2 


Summer 2007 


2007-2008 


2008-2009 


Biology 


Summer 2007 


2007-2008 


2008-2009 



WestEd originally recruited 86 history teachers and randomly assigned them to 
treatment and control. However, there are currently 48 teachers remaining in the study from 
History Cohort 1. The drop in participation was due primarily to a range of external factors 
including subject reassignment and teachers leaving the profession. Several teachers did 
report the time commitment was too great, while four teachers provided no explanation. Of 
the 48 remaining teachers, 21 control and 16 treatment teachers returned at least one teacher 
assignment. In total, we received 36 Immigration, Industrialization, and Urbanization 
assignments (19 control and 17 treatment) and 36 World War II assignments (20 control and 
16 treatment). 

Scoring Protocol 

Two CRESST associate researchers and one UCLA History Ph.D. candidate scored the 
teacher assignments. These expert raters had used a similar rubric to score over 100 biology 
assignments in 2007. The CRESST researchers began scoring history assignments in January 
2008. They began by first reviewing the rubric and discussing the criteria for each dimension. 



to 




The raters then scored five assignments independently using score sheets that provide space 
for one numeric score and comments per dimension. These independently scored 
assignments served as the basis for discussing strategies for scoring each dimension (e.g., 
which aspects of the coversheet or submitted student work were proving to be most helpful in 
scoring a given dimension) and allowed the raters to establish final calibrated scores for all 
dimensions. The raters repeated this process when the third rater began scoring in April 2008. 

Once scoring was underway, raters discussed one-point discrepancies and then came to 
consensus on final score points. Raters relied heavily on the comments recorded in the score 
sheets to provide evidence for determining final scores. Additionally, they reviewed sections 
of the assignments during these discussions. At least two expert raters scored each of the 
assignments. A third rater resolved discrepancies greater than one score point. This rating 
process did not include averaging rater scores. Raters were either in exact agreement or 
negotiated a final score. The purpose of this process is to support high reliability by 
developing expert raters. 

Results 

This section provides a statistical summary of two central goals of the Teacher 
Assignment instrument development, namely that the products of our work would be 
measures that could be easily and consistently scored by raters with moderate content 
knowledge, and that these measures would be sensitive to the curricular intervention. 

Rater Reliability 

The Intra-class Correlation (ICC) was computed to measure inter-rater reliability of all 
rubric dimensions. The ICC is a measure of the variability within raters as a proportion 
(reported in decimal form, from zero to one) of the total variation across all ratings and all 
subjects (Shrout & Fleiss, 1979). In the case of perfect agreement, 100% of the variation is 
accounted for within raters, and the ICC equals 1. As seen in Table 2, for all dimensions, the 
average inter-rater reliability was outstanding (> 0.8), or substantial (0.6 to 0.79; see Landis 
& Koch, 1977). 
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Table 2 

Inter-rater Reliabilities 



Rubric Dimension 


Immigration, 

Industrialization, 

Urbanization 


World War II 


Reading Opportunities 


0.77 


0.61 


Reading Comprehension Strategies 


0.91 


0.90 


Metacognitive Processes 


0.88 


0.97 


Disciplinary Reading 


0.93 


0.86 


Collaborative Meaning Making 


0.85 


0.89 


Teacher Instruction: Support For Reading Engagement 


0.87 


0.91 


Teacher Instruction: Accommodations For Reading 


0.94 


0.97 


Cognitive challenge 


0.72 


0.75 


Teacher Instruction: Support For Cognitive Challenge 


0.76 


0.78 


Monitoring: Adjusting Instruction 


0.82 


0.90 


Monitoring: Student Feedback 


0.87 


0.84 



In a simple intent-to-treat comparison of treatment and control (averaging teacher 
scores over the two assignments), treatment teachers significantly outscored control teachers 
on six dimensions: (a) reading opportunities; (b) reading comprehension strategies; 
(c) metacognitive processes; (d) collaborative meaning making; (e) teacher instruction: 
support for reading engagement; and (f) student feedback (p < .05). If, instead of averaging 
the scores across the two assignments, we examined the two assignments independently, the 
findings were highly consistent. Differences appeared in most cases on the same dimensions 
for both assignments, with similar magnitude. The two notable exceptions were on the 
Disciplinary Reading construct, for which the observed difference was only significant on the 
second assignment and the Monitoring: Student Feedback construct, for which the findings 
were not statistically significant in the smaller samples. 

As seen in Table 3, overall, the highest treatment assignment mean scores were for the 
reading opportunities (3.69), support for reading engagement (3.56), and reading 
comprehension strategies (3.38) dimensions. The highest control assignment mean scores 
were for reading opportunities (3.05), support for cognitive challenge (2.95), and cognitive 
challenge (2.76). The reading opportunities scores were high because nearly all teachers 
followed the data collection requirements and submitted assignments with at least one 
reading opportunity. 
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On the other end of the spectrum, the lowest treatment assignment mean scores were 
for adjusting instruction (1.59), accommodations for reading (1.91), and Disciplinary 
Reading (2.47). Among controls, scores were lowest for metacognitive processes (1.26), 
collaborative meaning making (1.52), and accommodations for reading (1.60). Low scores 
for reading accommodations suggest that history teachers without RA training are unlikely to 
differentiate instruction according to differences in student reading skills. 



Table 3 

Means and Standard Deviation of Teacher Assignment Scores by Treatment Status 



Rubric Dimension 


Type of 
Teachers 


N 


Mean 


Std. 

Deviation 


Std. 

Error 

Mean 


Reading Opportunities* 


Control 


21 


3.05 


0.63 


0.14 




Treatment 


16 


3.69 


0.36 


0.09 


Reading Comprehension Strategies* 


Control 


21 


1.88 


0.67 


0.15 




Treatment 


16 


3.38 


0.70 


0.17 


Metacognitive Processes* 


Control 


21 


1.26 


0.44 


0.10 




Treatment 


16 


2.66 


0.91 


0.23 


Disciplinary Reading 


Control 


21 


1.93 


0.64 


0.14 




Treatment 


16 


2.47 


1.18 


0.29 


Collaborative Meaning Making* 


Control 


21 


1.52 


0.64 


0.14 




Treatment 


16 


2.59 


1.00 


0.25 


Teacher Instruction: Support For Reading 
Engagement* 


Control 


21 


2.00 


0.74 


0.16 




Treatment 


16 


3.56 


0.70 


0.18 


Teacher Instruction: Accommodations For 
Reading 


Control 


21 


1.60 


0.74 


0.16 




Treatment 


16 


1.91 


0.99 


0.25 


Cognitive Challenge 


Control 


21 


2.76 


0.58 


0.13 




Treatment 


16 


2.72 


0.98 


0.25 


Teacher Instruction: Support For Cognitive 
Challenge 


Control 


21 


2.95 


0.52 


0.11 




Treatment 


16 


2.56 


0.79 


0.20 


Monitoring: Adjusting Instruction 


Control 


21 


1.69 


0.94 


0.21 




Treatment 


16 


1.59 


0.80 


0.20 


Monitoring: Student Feedback* 


Control 


21 


2.40 


0.68 


0.15 




Treatment 


16 


2.94 


0.87 


0.22 



Note. *p < .05 for independent samples t-test. 
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Conclusion 



Utilizing teacher assignments has proven to be a successful method of measuring 
teacher practice as demonstrated by the high inter-rater reliability of all 1 1 rubric dimensions 
and sensitivity to instruction demonstrated by the observed differences between treatment 
and control classrooms. Whereas earlier versions of teacher assignment rubrics focused on 
evaluating the general quality of instruction, this adapted rubric detected specific elements of 
a professional development program. 

History Cohort 1 treatment teachers significantly outscored control teachers on six 
dimensions (reading opportunities; reading comprehension strategies; metacognitive 
processes; collaborative meaning making; teacher instruction: support for reading 
engagement; and monitoring student feedback) that are critical to RA implementation. 
Overall, there is strong evidence demonstrating that teachers who participated in professional 
development program are incorporating RA aspects into their teaching practices. The high 
inter-rater reliability combined with the significant differences between study groups 
suggests the new instrument can successfully detect the impact of a specific professional 
development program on teacher practice. 

Evidence from CRESST’s ongoing work with a second cohort of history teachers, as 
well as a parallel study of biology teachers, will be completed later in 2009, and should hold 
further clues about the robustness and generalizability of these findings. Early indications 
from this work are positive, and will be detailed in a forthcoming technical report related to 
the student outcomes measures and the relationship between teacher instruction and student 
achievement. 
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WestEd 

Teacher Assignment Collection 

In-class Assignment 

History 

Immigration, Industrialization, and Urbanization 



Teacher ID 




Teacher Name 




School Name 




Type of U.S. History Class 




Student Grade Span 




Period 




Date of This Assignment 





CRESST 

National Center for Research on Evaluation, Standards, and Student Testing 
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ASSIGNMENT COVER SHEET: HISTORY 
IMMIGRATION, INDUSTRIALIZATIN, AND URBANIZATION 



1. Content Standards 

Which Content Standards in U.S. History were you targeting for this assignment? 
Please list standards below. 



2. Assignment Description 

a. Describe the assignment in detail 
What were students asked to do? 

Please attach a copy of the assignment directions you distnbuted to students. 

Please attach copies of any supporting matenals you distributed to students (teacher notes, handouts, etc ). 



Page 2 of 8 
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Teacher Assignment Collection: History- Immigration, Industrialization, and Urbanization 



b. How long was the class period in which students worked on this assignment? 

It the assignment was completed in class, how many minutes did students take to complete it? 

it the assignment was not completed by the end of the class, what percentage of the assignment was 
done in class and what percentage was done at home? In class: At home: 

c. Approximately how many assignments like this do you give per year? 

d. What percentage ot the assignment was completed individually? 

e. What percentage was completed in pairs? In groups? 

t. Did students participate in a small group and/or whole class discussion? □ small group □ whole class 

g. It students worked together on any part of this assignment, for example with partners, in small groups, or 
with the whole class, please describe the nature ot their collaboration. 



Page 3 of 8 
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Teacher Assignment Collection History - Immigration, Industrialization, and Urbanization 



3. Reading Material Information 

a. What kinds ot reading material did you use tor this assignment (e g., textbook, article, graphs, maps, 
photographs or other primary source documents, etc). Please indicate the text type, source, title, author, number ot 
pages and reading level d any material students read as part of this in-class assignment. To indicate the reading level 
of the material, please write whether you felt the text was below, at, or above your students’ grade level. 



Please include an appropriate sample section from the text. 



Text Type 


Sources 


Text Title 


Authors/Publishers 


# of Pages 
Assigned 


Reading Level 















b. Explain why you chose this reading material tor this assignment. 



c. If students engaged in any intertextual readings for this assignment (i.e., comparing views and information 
from different text sources) please describe this below. 



Page 4 of 8 
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Teacher Assignment Collection: History- Immigration, Industrialization, and Urbanization 



4. Learning Routines and Tools 

Please list and describe any literacy routines, reasoning processes, and comprehension strategies that were 
used during this assignment. If any learning tools, such as logs, notes, or graphic organizers were used as part of this 
assignment, please attach a copy. 



5. Metacognitive Processes 

Describe in detail any metacognitive processes students engaged in for this assignment (e g., annotating in text 
margins, thinking aloud, completing metacognitive reading logs, and discussion focusing on students’ thinking and 
problem-solving processes). 



Page 5 of 8 
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Teacher Assignment Collection: History- Immigration, Industrialization, and Urbanization 



6. Description of Instructional Strategies 

a. Describe in detail the instructional strategies you used for the history and literacy components of this assignment 
(e g, modeling, explicit instruction, presentations, discussion opportunities, detailed sequences of work, etc ). 



b. If you did anything specific to support collaboration and/or student engagement during this assignment, please 
describe this below. 



c. Describe why you chose these particular instructional strategies for this assignment. How were these instructional 
strategies and methods intended to help your students accomplish the history and literacy goals of this assignment? 



Page 6 of 8 
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Teacher Assignment Collection: History - Immigration, Industrialization, and Urbanization 



d. What is the percentage of ELLs in your classroom? 

e. What is the percentage of struggling readers in your classroom? 

If you differentiated instruction for students such as ELL and struggling readers, please provide a detailed 
description below. 



7. Monitoring and Assessment 

a. How would you rate the challenge level of this assignment tor the majority of your students? 

High Challenge Medium Challenge Low Challenge 

b. What proportion of your class struggled during the assignment? ==== % 

c. How did you monitor student performance during the assignment? Please describe. 



Page 7 of 8 
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Teacher Assignment Collection: History- Immigration, Industnalization, and Urbanization 



d. If you made any changes to your curriculum, instruction, or lesson plan based on monitoring student 
performance, 

please describe below. 



e. Did you provide feedback to your students during and/or after the assignment? Please describe. 



f. Approximately what percentage of the students in your class performed at the following levels by the end of this 
assignment? 



% = Good to Excellent 



% = Adequate 



% = Not Yet Adequate 



Page 8 of 8 
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The purpose of this dimension is to evaluate the degree to which the teacher used this assignment as a 
vehicle to proside students with the opportunity to read. Qualities of reading opportunities include: 
the role of reading, duration of reading, and text variety (e.g.. essays, arguments, and primary 
documents such as letters, newspapers, and comic strips). Evidence for this dimension can be found 
throughout the coversheet, particularly in response to questions 2a, 2b. 3a and 3b. 



The teacher provides substantial opportunities for meaningful reading experiences 

4 as demonstrated by: 

• The role of history reading is central to the assignment with little apparent 
boundary between reading and other tasks. Reading is necessary for completing 
the assignment. The text(s) is clearly related to the standards, knowledge, and or 
skills targeted by the assignment. 

• Significant time is set aside for reading history texts with oppoitunities for 
recursive readings of shorter/easier texts and or in-depth reading of longer more 
difficult texts. 

• A variety of texts may used. 

3 The teacher provided adequate opportunities for reading as demonstrated by: 

• The role of history reading is supportive to the assignment overall (e.g. students 
read during the mput phase and then move on to a hands-on task for the remainder 
of the assignment). Reading is necessary for completing the assignment. The 
text(s) is mostly related to the standards, knowledge, and/or skills targeted by the 
assignment. 

• Adequate time is set aside for reading history texts. 

• A variety of texts may used. 

2 The teacher provided minimal oppoitunities for reading as demonstrated by: 

• The role of history reading is supplemental to the assignment overall (e.g. the 
reading task is an add-on). Reading is not necessary for completing the 
assignment. The text(s) is somewhat related to the standards, knowledge, and or 
skills targeted by the assignment. 

• Time allotted for reading may be brief or allow for only superficial reading. 

• There is likely little text variety. 



The teacher provided no opportunities for reading as part of this assignment and or 
there is not enough evidence to make a judgment. 
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Reading Comprehension Strategies 



The purpose of this dimension is to describe the degree to which the teacher provided students 
with the opportunity to utilize various comprehension strategies to assist in their comprehension 
of history reading. These reading comprehension strategies include: generating questions, 
previewing text organization, and using reading logs. Evidence for this dimension can be found 
throughout the coversheet. particularly in response to question 4. 



The teacher provided students with substantial opportunities to utilize reading 

4 comprehension strategies as demonstrated by: 

• Teacher described the role of the reading comprehension strategies in the cover 
sheet (e.g.. teacher considers this to be a formal part of the assignment). 

• Teacher allocated sufficient time during the assignment for students to use 
various reading comprehension strategies. 

• Reading strategies are targeted and purposeful (i.e.. teacher tailored the reading 
strategies for the specific texts used in the assignment). 

• Teacher held students accountable for using reading comprehension strategies 
(e.g.. reviewed reading logs, circulated while students talk-to-the-text. etc.). 

The teacher provided students with adequate opportunities to utilize reading 

3 comprehension strategies as evidence by: 

• Teacher made mention of reading comprehension strategies in the cover sheet. 

• Teacher allocated sufficient time during the assi gnm ent for students to use 
various reading comprehension strategies. 

• Reading strategies may be general and/or part of an ongoing reading routine 
(e.g.. pair share, use reading logs). 

2 The teacher provided few opportunities for students to utilize reading 
comprehension strategies as demonstrated by: 

• Teacher may not have mentioned reading strategies in the coversheet. but there 
is evidence of their use in sample of student work. 

• Teacher allocated limited time during the assignment for shidents to use 
reading comprehension strategies. 

• Types of reading strategies used may be vague. 

I Tlie teacher did not provide opportunities for students to utilize reading 
comprehension strategies hi the context of tins assignment or there was not 
enough evidence to make a judgment. 
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Meta cognitive Processes 



The purpose of this dimension is to evaluate the degree to which the teacher used this assignment as a 
vehicle to provide students with the opportunity to utilize various metacoguifive processes such as 
annotating in text margins, thinking aloud, completing metacoguifive reading logs, and conversation 
focused on students’ thinking and problem solving processes. Evidence for this dimension can be found 
throughout the coversheet, particularly ill response to questions 5, 6a. and samples of student w ork. 



4 The teacher provided students with significant opportunities within the context of 
this assignment to utilize metacognitive processes as demonstrated by: 

• Students identified confusions and or new understandings (e.g.. talking-to-the- 
text). 

• Students are required to reflect and self-evaluate their comprehension levels 
(e.g.. group conversations, reading journals, etc.). 

• Smdents made individualized adjustments based on their self-evaluations (e.g.. 
review difficult passage with a peer, use an additional reading strategy, etc.) 
and/or teacher provided opportunities for students to make adjustments (e.g.. 
teacher provides students with additional internet resources and or encyclopedia 
to clarify confusions in existmg text set). 



3 The teacher provided students with adequate opportunities within the context of 
this assignment to utilize metacognitive processes as demonstrated by: 

• Students identified confusions and/or new understandings. 

• Smdents reflected and self-evaluated their comprehension levels, but this step 
may not be formally structured (e.g.. could be evidence in student work only and 
not in assignment cover sheet). 




The teacher provided smdents with minimal opportunities within the context of 
this assignment to utilize metacognitive processes as demonstrated by: 

• Smdents identified confusions and/or new understandings, but process for doing 
so may not be formally structured. 

The teacher did not provide smdents any opportunities within the context of this 
assiginnent to utilize metacognitive processes or there was not enough evidence to 
make a judgment (e.g.. teacher writes that there was a class discussion but 
provides no additional details articulating how this discussion was metacognitive 
in nature). 
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Disciplinary Reading 



This dimension considers the degree to which the teacher used this assignment as a vehicle to 
proside students with the opportunity to utilize Disciplinary Reading processes such as 
conducting iutertextiial readings (e.g., comparing and contrasting texts — including maps, graphs, 
history symbols, as well as written text), analysis of text discourse structures in history texts (e.g., 
reading headings and subheadings to determine text organization), evaluating the source of a 
document, identifying the perspective or point of view taken, and placing the primary source 
document into a historical context (contextualizing). Evidence for this dimension can be found 
throughout the coversheet, particularly in response to questions 2a and 3c. 



4 



3 

2 

1 



Within the context of tins assignment, the teacher provided students with significant 
opportunities to utilize Disciplinary Reading processes. These opportunities include at least 
two of the following activities conducted as a significant aspect of the assignment or three of 
the activities conducted in a more limited maimer. * 

• evaluating the source of a text 

• identifying the perspective or point of view of a text 

• conducting intertextual readings (e.g.. comparing information presented in texts of the same 

subject matter) 

• placing a primary source document into a historical context 

• analyzing and interpreting cause and effect 
and/or 

• understanding the text in relation to the major debates among historians. 

Within the context of this assignment, the teacher provided students with adequate 
opportunities to utilize Disciplinary Reading processes. These opportunities include at least 
one of the activities above conducted as a significant aspect of the assignment or two of the 
activities conducted in a more limited maimer. 

The teacher provided students with minimal opportunities to utilize Disciplinary Reading 
processes as part of this assignment. The teacher may have asked students to participate in 
one of the above mentioned activities in a limited manner (e.g.. not as a significant aspect of 
the assignment). 

Hie teacher did not have provided students with any opportunities to utilize Disciplinary 
Reading processes as part of this assignment or there was not enough evidence to make a 
judgment. 



* an assignment requiring students to consider the text in relation to the major debates among 
historians automatically receives a 4. 
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COLLABORA TIVE MEANING MAKING (DISCUSSION OPPORTUNITIESi 



The purpose of this dimension is to describe the degree to which the teacher used this assignment 
as a vehicle to proside students with the opportunity to participate in discussions with peers 
focused ou history texts. This dimension also considers the opportunity teachers provided 
students to read in small or paired grouping configurations. Evidence for this dimension can be 
found throughout the coversheet, particularly in response to questions 2d. 2e. 2f. 2g. and 6b. 



4 The teacher used this assignment as a vehicle to provide students with significant 

opportunities to participate hi discussions with peers focused on history texts as demonstrated 

1 by: 

• The collaborative work is well-structured (i.e.. the teacher communicates a clear purpose to 
students). 

• A strong routine is in place to support the collaboration (e.g.. collaboration was a required 
part of assignment, the teacher communicates explicit directions to students, may provide 
groups with graphic organizers, etc.). 

• There is accountability for the collaborative meaning making at the individual level (e.g.. 
each student is responsible for documenting, presenting, etc. some aspect of the 
collaborative work). 

• The collaborative work is directly and strongly connected to the next step in the overall 
assignment. 

3 The teacher used this assignment as a vehicle to provide students with adequate opportunities 
to participate in discussions with peers focused on history texts as demonstrated by: 

• The collaborative work is adequately structured (i.e.. there is a clear piupose for the 
collaboration, but unclear whether or not teacher communicates this to students). 

• An adequate routine is in place to support the collaboration (e.g.. collaboration was a 
requir ed part of assignment, teacher instructions are mostly clear). 

• There is accountability for the collaborative meaning making at the group level (e.g.. 
groups are responsible for sharing out. using group conclusions for the next part of the 
assignment, etc.). 

• The collaborative work somewhat connects to the next step in the assignment. 

2 The teacher provided students with minimal opportunities to participate in discussions with 
peers focused on history texts. The collaboration was a required part of the assignment, but 
lacked structure and may not have been supported by a routine. 

I The teacher provided students with no required opportunities to participate in discussions 
with peei-s focused on history texts. If collaborative meaning making did occur, it was 
suggested as optional and/or was student-driven (e.g.. students were allowed to help each other 
if they wanted to with little or no direction from the teacher). 
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Teacher Instruction: Support for Reading Engagement 



The purpose of this dimension is to evaluate the degree to which a teacher supports students in their 
successful completion of the reading task. Specifically, this dimension considers literacy support 
activities such as whether the teacher models (i.e.. demonstrates an aspect of the reading process). 
provides explicit instruction (i.e.. articulates the various steps and/or processes students required of the 
reading task), provides resources (e.g.. consumable texts, graphic organizers), and or establishes 
literacy routines (i.e.. puts in place ongoing and specific reading process practices). Teacher support 
for reading engagement may focus on any aspect of the reading process (e.g.. reading logs, focused 
conversations, reading comprehension strategies, metacognitive activities, etc.). Evidence for this 
dimension can be found throughout the coversheet. particularly in response to questions 4. 5. 6a. 6b. 
and 6c. 



4 Teachers significantly supported students in the reading task through previous or current 
teaching approaches. This support includes at least two of the following support types in a 
significant way or three of the following support types in a more limited maimer. 

• Teacher modeled a specific pait or parts of the reading process. 

• Teacher provided explicit instruction around the reading task and/or process. 

• Teacher provided resources to support reading task. 

• Teacher established clear literacy routines. 

Additionally, the reading task is well-structured (e.g.. broken down into a series of steps and 
well-scaffolded). 

3 Teachers adequately supported students in the reading task. This support includes at least 

one of the following support types in a significant way or two of the following support types 
in a more limited manner. 

• Teacher modeled a part or parts of the reading process. 

• Teacher provided explicit instruction around the reading task and or process. 

• Teacher provided resources to support reading task. 

• Teacher established literacy routines. 

Additionally, the reading task is mostly well-structured (e.g.. structure is sound overall, but 
some elements may be unclear). 

2 Teachers somewhat supported students in the reading task. 

• The teacher incorporated one type of support in a limited maimer. 

• The reading task was not appropriately structured and/or was unclear. 

Teachers did not support students in the reading task or evidence was too vague to make a 
judgment. The reading task was not structured. 
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Teacher Instruction: Accommodations for Reading 



The purpose of this dimension is to describe the degree to which a teacher tailored the assignment to 
meet the various reading needs of his/her students. Specifically, this dimension considers whether the 
teacher differentiated instruction though accommodations such as: providing various texts for students 
to read at different reading levels: providing extra support for struggling readers and ELs. e.g.. by 
modifying instruction, giving help outside of class and adapting the assignment content: allowing 
students to work at their own pace: and paring struggling reading with stronger readers. Evidence for 
this dimension can be found throughout the coversheet. especially in response to questions 3a. 3b. and 
6d. 



4 The teacher significantly tailored the assignment to meet the various reading needs of 
his/her students. Specifically, the teacher differentiated instruction using at least two of 
the following methods and described the accommodation with clarity and specificity: 

• providing various texts for students to read at different reading levels 
• providing extra teacher support for struggling readers and ELs. (e.g.. by modifying 
instruction, giving help outside of class, adapting the assignment content) 

• allowing students to work at then own pace 
• pairing snuggling readers with stronger readers 

3 The teacher adequately tailored the assignment to meet the various reading needs of 

his/her students. Specifically, the teacher provided some differentiated instruction using 
at least one of the following methods and described the accommodation in a mostly 
clear and specific manner: 

• providing various texts for students to read at different reading levels 
• providing extra teacher support for struggling readers and ELs. (e.g.. by modifying 
instruction, giving help outside of class, adapting the assignment content) 

• allowing students to work at then own pace 
• pairing struggling readers with stronger readers 

2 The teacher minimally tailored the assignment to meet the various reading needs of 

his/her students. Specifically, the teacher provided some differentiated instruction using 
at least one of the methods mentioned above, but described the accommodation with 
insufficient clarity and specificity. 

The teacher did not tailor the assignment to meet the various reading needs of his/her 
students. 
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Cognitive Challenge 



The purpose of this dimension is to describe the degree to which teachers required students to 
apply complex cognitive skills when engaging with history concepts in this assignment. The 
dimension also considers the level of critical thinking teachers required of the students in order to 
complete the assignment (e.g., critical thinking, problem solving, analyzing, and synthesizing 
information). Specifically, this dimension considers the opportunity teachers provided students to 
construct or transform knowledge as opposed to simply recalling, describing, or identifying basic 
information. Evidence for this dimension can be found throughout the coversheet. especially in 
response to question 2a. the assignment instructions given to students (if included), and samples 
of student work. 



4 The teacher required students to significantly utilize higher-order thinking skills by 
engaging in reasoning processes such as analysis, synthesis, and or evaluation of 
historical concepts in order to complete the assignment. The higher-order thinking 
processes are the means by which deep content understanding is acquired. 




The teacher required students to utilize some higher-order thinking skills by 
engaging in reasoning processes such as the application and/or analysis of 
historical concepts in order to complete the assignment. These thinking processes 
are the means by which adequate content understanding is acquired. 



The teacher required students to utilize basic comprehension skills such as the 
explanation, description, and/or identification of historical concepts in order to 
complete the assignment. The lower-level thinking processes are the means by 
which surface content understanding is acquired. 



I Tlie teacher required students to utilize only basic knowledge and lower-level 
thinking skills such as knowledge recall, definition, labeling, and/or listing of 
historical concepts in order to complete the assignment. The lower-level thinking 
processes are the means by which only minimal content understanding is 
acquired. 
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Teacher Instruction: Support for Cognitive Challenge 



The pui pose of this dimension is to describe the degree and qualitv of support a teacher provides for the 
assignment’s cognitive challenge. Specificallv, this dimension considers the degree of support for the 
thinking skills (e.g.. knowledge, comprehension, application, aualvsis, synthesis, and or evaluation) and 
processes that are provided by the teacher for successful completion of the assignment. An assignment 
given a high score on this dimension had to have provided support that was focused on the cognitive task 
students were to carrv out; additionally, it will most likelv have a high percentage of students performing 
at ail adequate level or above. Evidence for support will be provided by the student samples and the 
description in the cover sheet, primarily in the “Description of Instructional Strategies” (section 6) but 
may also be found in any other section of the cover sheet (esp. 7a and 7d). 



4 

3 

2 

1 



Students are i veil supported in meeting the cognitive challenge of the assignment through previous or 

current teaching approaches such as: 

• Teaching of thmkmg processes (e.g., modeling, class discussions). 

• Structumig of the cognitive activity into an appropriate number of explicit steps (e.g., 
exposure to, application of, and analysis of concepts). 

• Enabling students to draw on peer or expert knowledge to work through history content 
(e.g., pair or group discussion). 

• Makmg resources available and reviewing them with students to aid in meetmg the 
cognitive challenge of the assignment (e.g., samples of student work with cntical 
thmkmg processes made exphcit). 

Students are adequately supported m meetmg the cognitive challenge of the assignment. 

• The teacher provides — or may have previously provided — students with adequate support for facilitatmg the 
necessary thmkmg skills. 

• The activity is fairly well structured mto explicit steps. 

• Students may also be provided with the opportunity to draw on peer knowledge (e.g.. pan or group discussion). 

• Resources may be provided to aid m meetmg the cognitive challenge of the assignment, but are not necessarily 
reviewed as a class (e.g., samples of thmkmg processes m student work are provided to students: students are 
expected to take the initiative to use the samples of student work without dnection from the teacher). 

Students are somen hat supported m meetmg the cognitive challenge of the assignment. 

• The teacher may provide - or may have previously provided - minimal support for facilitatmg thmkmg 
skills. 

• The activity may not be well structured mto explicit steps. 

• Teacher mentions makmg resources available, but it is unclear what the resources were and or how they 
were to be used. 

Students are generally not supported in meetmg the cognitive challenge of the assignment. 

• The teacher either does not provide students with any information on thmkmg processes through previous 
or current teaching approaches, does not mcorporate tmie mto the assignment for their use, or both. 

• The activity is not structured mto explicit steps. 

• No resources were provided to aid in meetmg the cognitive challenge of the assignment. 
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Monitoring: Adjusting Instruction 



The purpose of this dimension is to capture the degree to which the teacher adjusts instruction 
based on monitoring student progress. Specifically, this dimension considers whether the teacher 
made curricular, instructional, or lesson adjustments for the immediate benefit of the current 
students. Evidence for this dimension can be found throughout the coversheet. especially in 
response to questions 7b, 7c and 7d. 



4 The teacher monitors student progress and makes specific adjustments to 
instruction for the immediate benefit of current students (i.e.. the teacher 
adjusts instruction dining the course of the assignment). 



3 The teacher monitors student progress and makes general adjustments to 
instruction for the benefit of current students, but may not make adjustments 
immediately (e.g.. teacher may implement changes after the assignment has 
been completed). 



2 The teacher monitors student progress and instructional adjustments are 
unclear and or adjustments are intended for future students (e.g.. teacher 
describes lesson plan adjustments to be made next school year). 

I Tlie teacher does not use the assignment as an opportunity to monitor student 
progress for the purpose of adjusting instruction. 
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The purpose of this (limeusiou is to capture the degree to which the teacher provides students 
with feedback. Specifically, this dimension considers whether teachers provided feedback to 
positively impact student performance. Evidence for this dimension can be found throughout the 
coversheet, especially in response to questions 7b, 7c and 7e, and samples of student work. 



4 The teacher provides specific and critical (necessary information for 
successful task completion) feedback to smdents during and or after the 
assignment that is intended to improve student learning. 




The teacher provides general feedback to students during and or after 
the assignment that may improve student learning. 




The teacher provides minimal or unclear feedback to students during 
and or after the assignment. It is unlikely that this feedback will impact 
student learning. 



The teacher does not provide smdents with any feedback at any pomt 
during the assignment. 
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